Skip to content

Wrap websocket flush in timeout#2908

Merged
joshua-spacetime merged 3 commits into
masterfrom
joshua/fix/ws-client-actor
Jul 1, 2025
Merged

Wrap websocket flush in timeout#2908
joshua-spacetime merged 3 commits into
masterfrom
joshua/fix/ws-client-actor

Conversation

@joshua-spacetime

@joshua-spacetime joshua-spacetime commented Jun 27, 2025

Copy link
Copy Markdown
Contributor

Description of Changes

This patch:

  1. Breaks out of the websocket client actor loop on a close frame
  2. Adds comments as to why this is safe to do
  3. Wraps websocket flushes in a timeout, breaking out of the loop on timeout
  4. Adds comments
  5. Adds logging

The main change here is 3. Because if the flush stalls for any reason, before this patch, we could wait indefinitely without polling the other futures in the select! arms such as the liveness tick and the websocket receiver.

The flush could stall if the receiver stops ack'ing packets or if the tcp send buffer is full.

API and ABI breaking changes

None

Expected complexity level and risk

The change itself is not complicated, but the select! loop is. I'm fairly certain though that this change won't have any unintended consequences. It certainly doesn't poll fewer futures - it polls more.

Testing

Ran a large scale performance test and observed these websocket actors being dropped and cleaned up on disconnects (initiated from both server and client)

@joshua-spacetime joshua-spacetime force-pushed the joshua/fix/ws-client-actor branch 4 times, most recently from a64311f to 8156c8c Compare June 27, 2025 22:02
@joshua-spacetime joshua-spacetime marked this pull request as ready for review June 27, 2025 22:11
@joshua-spacetime joshua-spacetime force-pushed the joshua/fix/ws-client-actor branch from 8156c8c to 8b78173 Compare June 27, 2025 22:17
@joshua-spacetime joshua-spacetime changed the title Add more logging and break out of loop on close frame Wrap websocket flush in timeout Jun 27, 2025
@joshua-spacetime joshua-spacetime self-assigned this Jun 27, 2025
Comment thread crates/client-api/src/routes/subscribe.rs Outdated
Comment thread crates/client-api/src/routes/subscribe.rs Outdated
Comment thread crates/client-api/src/routes/subscribe.rs Outdated
Comment thread crates/client-api/src/routes/subscribe.rs Outdated
@bfops bfops added the release-any Can land in any release window. Will not block a release deployment. label Jun 30, 2025
@cloutiertyler

Copy link
Copy Markdown
Contributor

New changes also look good to me, since they're mostly mechanical in nature. Would be great to test, but I think it falls under mechanical enough.

@joshua-spacetime joshua-spacetime added this pull request to the merge queue Jul 1, 2025
Merged via the queue into master with commit e4d5c18 Jul 1, 2025
19 checks passed
@joshua-spacetime joshua-spacetime deleted the joshua/fix/ws-client-actor branch July 1, 2025 02:09
joshua-spacetime added a commit that referenced this pull request Jul 1, 2025
joshua-spacetime added a commit that referenced this pull request Jul 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-any Can land in any release window. Will not block a release deployment.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants